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Field of the Invention 

[0001] The present invention relates generally to image 

processing and particularly to systems and methods for obtaining and 

modifying images extracted from remote sources. 

Background of the Invention 
[0002] Computer generated documents have historically, like 

their pre-computer counterparts, contained only text. However, 
during the past 5-10 years, as computer hardware has become more 
powerful, storage capacities have increased, and presentation 
software has become more sophisticated, users have begun to 
introduce images into documents and presentations. 

[0003] However, finding images appropriate for a particular 

presentation is often prohibitively difficult. Users can generate 
the images themselves, but this process requires that the users have 
advanced facility with an image generation program and the time 
necessary to produce an image of reasonable presentation quality. 
Users can use a pre -generated image, but locating appropriate images 
is extremely difficult and time consuming. Pre-existing Images can 
be modified, but a pixel by pixel modification process is often 
nearly as time consuming as generating the image from scratch. 
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[0004] What is needed is a mechanism for obtaining pre-existing 

images and converting them to a format that allows them to be easily 
modified. 

Summary of the Invention 

[0005] The present invention includes systems, methods, and 

computer readable media for locating, sorting, and modifying images 
for future use. A conversion engine submits search requests to an 
image search engine and accepts the received results. The conversion 
engine arranges the search results according to selected visual 
characteristics of the image. Editing tools can then convert the 
results of the image search into vector based images which can be 
easily manipulated and enable users to modify the re-ranked images 
for future use. 

Brief Description of the Drawings 
[0006] FIGURE 1 is a block diagram illustrating the interaction 

between a user system, a search engine and several content sites. 
[0007] FIGURE 2 is a block diagram illustrating a computer that 

acts as a user system 

[0008] FIGURE 3 is a block diagram illustrating a closer view of 

a memory of the user system, 

[0009] FIGURE 4 is a block diagram illustrating a closer view of 

a conversion engine . 

[0010] FIGURE 5 is a diagram illustrating an image before and 

after modification. 
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[0011] 



FIGURE 6 is a diagram illustrating an image which can be 



modified by a user. 



[0012] 



FIGURE 7 is a flow chart illustrating an overview of a 



process for generating presentation-ready images. 



[0013] 



FIGURE 8 is a flow chart illustrating an process for 



ranking image results according to ease of modification. 



[0014] 



FIGURE 9 is a flow chart illustrating a process for 



modifying converted images. 



Detailed Description 



[0015] 



The present invention includes systems, methods, and 



computer readable media for locating, sorting, and modifying images 
for future use. FIGURE 1 is a block diagram illustrating the 
interaction between a user system, a search engine and several 
content sites. A user system 105 is preferably a workstation at 
which a user is attempting to generate a document or presentation 
augmented with images. A query for images is submitted from the 
user system 105 to a search engine 110. 

[0016] The search engine 110 is a commercial or private search 

engine such as Google or Alta Vista, which maintains a catalogue of 
images. The images are indexed by description, size, content 
maturity, and any other applicable criteria. Alternately, the search 
engine 110 can be an internal search program which searches through 
internal search collections such as previously downloaded images or 
stored Power Point slides. The search engine 110 returns links to 
the images to the user system 105. The user system 105 downloads the 
links and retrieves the images from the content sites 115,120, 125. 
The user system 105 then converts the raw image files to a vector 
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based format. The user system 105 then reorders the images according 
to user selected criteria such as their ease of modif iability, image 
style, or similarity to previously downloaded images. Image 
qualities such as file type, histogram characteristics, number of 
colors, number and length of edges, and number and length of 
segments, number and area of image sections, and any other 
applicable features are used to determine modif lability . The user 
system 105 returns to the user modified search results based upon 
these criteria. In one embodiment, the user system 105 provides a 
transparent interface to the user, presenting the modified results 
to the user as if they were returned directly by the search engine 
110. The user system 105 then enables the user to modify the 
downloaded images and integrate them into documents and 
presentations . 

[0017] FIGURE 2 is a block diagram illustrating a computer that 
acts as a user system 105. The system includes a processor 202. 
There may be more than one processor 202. Also coupled to the bus 
204 are a memory 206, a storage device 208, a keyboard 210, a 
graphics adapter 212, a pointing device 214, a speaker 215, and a 
network adapter 216. A display 218 is coupled to the graphics 
adapter 212. 

[0018] The processor 2 02 may be any specific or general -purpose 

processor such as an INTEL x8 6 or POWERPC -compatible central 
processing unit (CPU) . The storage device 208 may be any device 
capable of holding large amounts of data, such as a hard drive, 
compact disk read-only memory (CD-ROM)', DVD, or some other form of 
fixed or removable storage device. 
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[0019] FIGURE 3 is a block diagram illustrating a closer view of 

a memory 206 of the user system. The memory includes a web browser 
310. The web browser 310 is conventional and is configured to enable 
users to navigate to search engines 110 and submit searches for text 
or images . 

[0020] A conversion engine 315 is configured to receive image 

search results received from the content sites 115,120, 125. 
Receiving the image may involve connecting to the search engine 
through a search engine API or intercepting results transmitted by 
the search engine. When the results of an image search are returned, 
the conversion engine 315 parses the links for the searched images 
and downloads the source images. The conversion engine 315 then 
converts the images to a modifiable format. The conversion engine 
315 may use a commercial vectorization package such as Free Soft 
Silhouette or Streamline from Adobe or a proprietary mechanism that 
converts raw images to a more easily modifiable format. 
[0021] The conversion engine 315 then reorders the search 

results according to predefined criteria such as an ease of 
modification, image type, similarity to previously retrieved images 
or some other assigned criteria. The image type can include a 
particular style of image (cartoon, drawing, etc) or other, 
previously defined criteria. Image qualities such as file type, 
histogram characteristics, number of colors, number and length of 
edges, number and length of segments, number and area of image 
sections, and any other applicable features are used by the 
conversion engine 315 to rank the results. 

[0022] The conversion engine 315 then provides the reordered 

results to the user. An editor module 325 is configured to modify 
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the results. The editor module 325 is configured to present the 
object in vector format to allow the edges and lines to be 
manipulated as objects rather than collections of pixels. In one 
embodiment, the images are represented as connected segments of 
Bezier curves. 

[0023] FIGURE 4 is a block diagram illustrating a closer view 

of a conversion engine. The conversion engine includes original 
search results 405 which are the hyperlink values and raw image 
files that are generated by an original user search. 

[0024] The conversion module 415 extracts the raw image files in 

the search results 4 05 which may be in BMP, JPG, GIF or any other 
format and converts the images to a modifiable format. The 
conversion engine 315 may use a commercial vector izat ion package 
such as Free Soft Silhouette or Streamline from Adobe or a 
proprietary mechanism that converts raw images to^ a more easily 
modifiable format . 

[0025] The reorder module 42 0 reorders the search results 

according to an ease of modif iability . Image qualities such as file 
type, histogram characteristics, number of colors, number and length 
of edges, number and length of segments, number and area of image 
sections, and any other applicable features are used by the reorder 
module 420 to organize the results. The reorder module 420 applies a 
weighted analysis, considering various modif iability indicators, as 
well as an image's ranking in the original search results. The 
weights assigned to various factors can be designated by a user, or 
preconf igured. The reorder module 42 0 stores the reordered search in 
the modified results 410. The modified results 410 are preferably 
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formatted searches that are configured to appear as search results 
presented by the search engine 110 . 

[0026] The conversion engine 315 also stores source subjects 430 

that are used to generate searches. The source subjects 430 gather 
information about documents that have recently been generated and 
store suggested searches based upon those documents. For example, if 
a user were preparing a Power Point presentation on cell phones, the 
source subjects would detect the Power Point presentation and 
suggest *^cell phone" as a potential title. The search information 
can be generated by a custom plug- in program that extracts text from 
slides as it is being generated and use the text in ongoing queries. 
[0027] FIGURE 5 is a diagram illustrating an image before and 

after modification. An unmodified image 505 is generated from a 
raw image file such as a JPG or GIF file stored in the search 
results 405. The conversion module 425 extracts the gathered images 
from the search results and stores it as a group of segments which 
are represented as vectors . 

[0028] The conversion module 425 then modifies the image 505 for 

ease of future manipulation. The conversion module 425 groups 
together segments that are parallel and close as single segments. 
For example, the conversion module 425 would recognize the outside 
edge 510 of the unmodified image 505 as part of a single border and 
convert the fragmented segments 510 into a single segment 524. 

[0029] The conversion module is also configured to recognize 

non- linear shapes and adjust them. The conversion module checks 
groups of associated fragments 512, and generates a best fit Bezier 
curve for the fragments. Thus collections of segments representing 
round objects 512, can be recognized as associated and replaced with 
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single whole segments 518. The modified images are then reranked and 
stored in the modified results 

[0030] FIGURE 6 is a diagram illustrating an image which can be 

edited by a user. The unedited image 605 has been stored in a 
vector-based format and has preferably been modified according to 
the procedure illustrated in FIGURE 5. Since the modified results 
410 store the images as collections of curve objects rather than 
pixel data, the editor module 325 is configured to allow for the 
adjustment of curve and line segments. By submitting a ^^draw over" 
segment 6 08 a user can select a segment to be modified. The editor 
module 325, by checking the location of the draw-over segment 608, 
determines the segment 605 to be modified. The user can then perform 
line-alteration operations on the segment and produce an edited 615 
version of the object 600 having an altered version 610 of the 
selected line 605. 

[0031] FIGURE 7 is a flow chart illustrating an overview of a 

process for generating presentation-ready images. The process begins 
with the conversion engine 315 receiving 705 a search from a user. 
In one embodiment, the source subjects 430 provide potential topics 
for the user. The conversion engine 315 then passes along 710 the 
search to the search engine 110, In one embodiment, the conversion 
engine includes an HTML parser that loads a home page for an image 
search engine such as Google and submits the search. In an alternate 
embodiment, the conversion engine 315 maintains a proprietary 
interface with the search engine 110. In yet another embodiment, the 
conversion engine 315 does not intercept the request. 

[0032] The conversion engine then receives 715 the results from 

the search engine in the form of links to images. As part of step 
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715 the conversion engine 715 downloads the linked images. The 
conversion engine 715 then reorders 720 the search results. The 
reordering process includes the conversion module transforming the 
images to a vector based format as indicated in FIGURE 9 . The 
process also entails the reorder module 43 0 performing a weighted 
analysis on the search results, as is described in FIGURE 8. The 
reordered images are then stored in the modified results 410. The 
editor module 325 then enables 725 a user to edit the returned 
images . 

[0033] FIGURE 8 is a flow chart illustrating a process for 

ranking image results according to a potential usability of the 
image . In one embodiment the images are ranked according to an ease 
of modif lability . In an alternate embodiment, the images are ranked 
according to a similarity to previously downloaded images, or a 
previously defined style of image. The process begins with the 
system determining 805 a file type for an image. The reorder module 
420 can check a file name extension or some other characteristic of 
the file. The reorder module 420 then determines 810 the histogram 
characteristics for the image. The reorder module 420 determines a 
number of intensity peaks for the image, with a lower number of 
intensity peaks indicating a greater ease of modif lability . The 
reorder module then determines 815 a number of colors for the image, 
with a lower number of colors indicating a greater ease of 
modif lability . 

[0034] The reorder module 420 then determines 820 a number of 

edges for the segment. A simple shape, such as a circle will have a 
low edge measurement, whereas complicated shapes like the outer edge 
of a facial drawing will have a high edge measurement. The number of 
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edges will depend partially upon the representation algorithm, with 
some algorithms able to represent complicated curves as single 
edges . 

[0035] The reorder module then determines 825 a number of 

segments, with a lower number indicating a higher ease of 
modif lability . As with edges, the number of segments will depend 
partially upon the representation algorithm, with some algorithms 
able to represent complicated curves as single segments. The reorder 
module 420 then ranks the image 83 0 based upon the factors above and 
the image's original search ranking. The reorder module 420 
preferably applies a weighted analysis to determine the ease of 
modif lability of the image or similarity to other images, with the 
weights either predetermined or user- selected . 

[0036] FIGURE 9 is a flow chart illustrating a process for 

modifying converted images. The conversion module 425 first 
applies an edge detection process 905 that determines an overall 
shape for the image. The conversion module then converts 910 the 
image to a group of vectors or curve equations that represent the 
image . 

[0037] The conversion module then cleans 915 the edges and 

curves of the image as per the process described in FIGURE 5. Edges 
that are near each other are joined as single edges. The image is 
then presented 920 to the user as a search result. 

[0038] Other features, aspects and objects of the invention can 

be obtained from a review of the figures and the claims. It is to 
be understood that other embodiments of the invention can be 
developed and fall within the spirit and scope of the invention and 
claims . 
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[0039] The foregoing description of preferred embodiments of the 

present invention has been provided for the purposes of illustration 
and description. It is not intended to be exhaustive or to limit 
the invention to the precise forms disclosed. Obviously, many 
modifications and variations will be apparent to the practitioner 
skilled in the art. The embodiments were chosen and described in 
order to best explain the principles of the invention and its 
practical application, thereby enabling others skilled in the art to 
understand the invention for various embodiments and with various 
modifications that are suited to the particular use contemplated. 
It is intended that the scope of the invention be defined by the 
following claims and their equivalence. 

[0040] In addition to an embodiment consisting of specifically 

designed integrated circuits or other electronics, the present 
invention may be conveniently implemented using a conventional 
general purpose or a specialized digital computer or microprocessor 
programmed according to the teachings of the present disclosure, as 
will be apparent to those skilled in the computer art. 
[0041] Appropriate software coding can readily be prepared by 

skilled programmers based on the teachings of the present 
disclosure, as will be apparent to those skilled in the software 
art. The invention may also be implemented by the preparation of 
application specific integrated circuits or by interconnecting an 
appropriate network of conventional component circuits, as will be 
readily apparent to those skilled in the art. 

[0042] The present invention includes a computer program product 

which is a storage medium (media) having instructions stored 
thereon/in which can be used to program a computer to perform any of 
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the processes of the present invention. The storage medium can 
include, but is not limited to, any type of disk including floppy 
disks, optical discs, DVD, CD-ROMs, microdrive, and magneto-optical 
disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs , VRAMs , flash memory 
devices, magnetic or optical cards, nanosystems (including molecular 
memory ICs) , or any type of media or device suitable for storing 
instructions and/ or data. 

[0043] Stored on any one of the computer readable medium 

(media) , the present invention includes software for controlling 
both the hardware of the general purpose/ specialized computer or 
microprocessor, and for enabling the computer or microprocessor to 
interact with a human user or other mechanism utilizing the results 
of the present invention. Such software may include, but is not 
limited to, device drivers, operating systems, and user 
applications . 

[0044] Included in the programming (software) of the 

general/specialized computer or microprocessor are software modules 
for implementing the teachings of the present invention. 
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